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Background 


*  Metrology  and  uncertainty  analysis  has  been 
through  many  updates  in  the  last  decade 

—  Guidance  from  ISO,  NIST,  NASA  and  AIAA  in  last  5  years 

*  Rarely  are  complete,  in-depth  uncertainty  analyses 
conducted  for  rocket  engine  measurements 

—  Comparing  changes  in  fuel  or  hardware  necessitates  a 
strong  understanding  of  uncertainties 

—  Additionally,  most  journals  require  some  analysis  for 
publication  (rarely  is  it  complete  or  in-depth,  however) 

*  Recent  upgrades  in  Edwards’  EC-1  small-scale  test 
facilities  have  utilized  an  involved  analysis 

—  Best-practices  were  developed  to  lower  uncertainty 

—  Process  upgrades  continue  as  the  analysis  gets  more  in- 
depth 

*  Here  the  best-practices  and  their  reasons  are 
presented 
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Metric  of  Interest 


*  Many  parameters  of  interest  could  be  used  as  a 
metric  in  the  uncertainty  analysis  of  a  small-scale 
facility 

*  c* -efficiency  is  chosen  here  because  of  its  utility  for 
a  variety  of  changes  (fuel,  injector,  cooling,  etc.) 

—  Measured  c*  is  PcAt/rhtota| 

—  Theoretical  c*  is  calculated  using  CEA  code  with  rocket 
selection  and  finite  chamber  area  option 

•  CEA  inputs  are  mixture  ratio,  area  ratio  (chamber  to  throat), 
formulation  of  hydrocarbon  (CxHy),  enthalpy  of  formation  of 
hydrocarbon,  chamber  pressure  and  reactant  temperature 

*  Each  input  to  the  measured  and  theoretical  c*  will  be 
discussed  in  terms  of  best  practices  and  minimizing 
uncertainty 

—  Includes  dependent  measurements  such  as  density  and 
vapor  pressure 
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Very  Quick  EC-1  Overview 


*  EC-1  facility  is  a  small-scale  engine  test  rig 

—  Typically  between  100-500  lb  thrust 
—  Heat-sink  oxygen-free  copper  hardware 

•  Heat  losses  and  their  accompanying  uncertainties  are  NOT 
considered  yet 

—  GOX  and  liquid  hydrocarbon  considered  here 

•  Also  can  run  with  gaseous  hydrogen  or  gaseous  hydrocarbon 
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Weighted  Least  Squares 


*  For  most  calibrations,  a  line  is  fit  to  the  calibration 
points 

—  The  method  used  can  be  quite  important  for  determining 
uncertainty  associated  with  linearity  and  hysteresis 

*  Simple  least  squares  methods  do  not  use  or  produce 
information  on  uncertainties 

*  Weighted  least  squares  methods  can  use 
uncertainty  inputs  and  calculate  uncertainties  in  the 
curve  fit 

—  A  good  method  will  consider  uncertainty  in  both  the 
supplied  and  measured  values 

—  A  good  method  will  not  assume  these  two  uncertainties 
are  identical 

—  A  good  method  will  allow  uncertainties  to  be  different  at 
each  point 

E.  Mathioulakis  and  V.  Belessiotis,  Uncertainty  and  Traceability  in  Calibration  by 
Comparison,  vol.  11,  no.  pp  771-775,  2000. 
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Pressure 


*  Selection  of  high-quality  transducers  is  imperative,  but 
not  always  easy  to  assure 

*  Several  problems  were  encountered  with  one 
manufacturer  but  not  with  another  manufacturer 


Female  bodies  had  zero  point  changes  if  fitting  was  tightened 

•  EC-1  moved  to  male  bodies  on  older  transducers;  new 
transducers  do  not  have  this  issue 

Internal  temperature  compensation 
not  effective 

•  EC-1  moved  to  uncompensated 
transducers  which  will  be 
calibrated  using  an  oven  to 
develop  temperature-dependence 

Manufacturer  given  uncertainty 
not  the  actual  uncertainty  of 

transducer  ‘D  5  10  tJ|(8) 

•  Calibration  is  a  must  to  determine  actual  uncertainty  esp. 
linearity  and  hysteresis  for  a  specific  transducer 
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Pressure 


*  Routine  calibration  of  transducers  is  required 


—  Prior  analysis  should  be  used  to  assure  transducers 
remain  within  the  uncertainty  and  uncertainty  should  be 
enlarged  if  the  calibration  period  was  too  long 


—  EC-1  calibrates  following  every  series  of  tests  (e.g.,  when 
an  injector  is  changed)  _ , _ _ _ _ _ 


—  A  series  of  upward  and 
downward  cycles  is  needed  2500 
to  elucidate  the  hysteresis  =.2000 
of  the  system  and  ensure  & 
its  inclusion  in  the  |  isoo 

uncertainty  ! 

1000 


—  Currently,  the  linearity  and 
hysteresis  of  the 
transducers  is  the  largest 
component  of  the  uncertainty 
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Pressure 


*  Match  the  transducer  to  the  range  being  recorded 

—  Using  a  0-3000  psi  transducer  to  measure  atmospheric 
pressure  might  give  a  reading  of  14  +/- 1  psi  while  a  0-15 
psi  transducer  with  a  similar  %  uncertainty  would  give  a 
13.72  +/-  0.015  psi  reading 

*  Atmospheric  pressure  should  be  recorded  daily  at 
minimum,  but  during  the  test  is  better 

—  Reduces  uncertainty  in  absolute  pressure  used  in  the  c* 
calculations 

*  Snubbing  with  a  low  density  gas  is  recommended 

—  Failing  this,  long  leads  and  periodic  inspection  are 

required  to  ensure  heat,  soot  and  unburned  hydrocarbons 
have  not  damaged  the  transducers 

—  Helps  insulate  transducer  from  soot  and  hydrocarbons 

—  Reduces  time  constant  even  with  reasonably  long  leads 

*  Proper  grounding  and  shielding  of  wires  is  crucial 
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Pressure 


*  Chopping  power  supplies  (regulators)  in  amplified 
transducers  creates  oscillatory  output 

—  Several  transducer  manufacturers  were  tried,  all  exhibited 
the  same  issue 

—  Issue  can  be  measured  with  oscilloscope  or,  depending  on 
DAQ  system,  may  be  visible  in  output  or  supplied  power 

*  Oscillations  feed  back  to  the  supply  system  and 
other  transducers 

*  EC-1  now  uses  unamplified  transducers 

*  Additionally,  power  is  supplied  via  DAQ  (Pacific 
Instruments  60-32-EM  card)  so  at  most  4  transducers 
are  powered  together 

*  With  changes,  no  oscillatory  behavior  is  observed 
and  standard  deviation  in  voltage  is  substantially 
decreased 
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Uncertainty  (psi) 


Pressure 


Two  randomly  selected  transducers.  New  transducers 

are  unamplified;  old  are  amplified 
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Temperature 


*  Temperature  enters  directly  in  CEA  input  along  with 
being  a  dependent  variable  in  both  flow  rates 

*  A  known  junction  temperature,  through  the  use  of  a 
junction  box,  lowers  uncertainty 

—  EC-1  uses  a  hot  junction  box  because  the  experimental 
cell  is  not  climate  controlled  (kept  at  150°F) 

*  Wire  length  should  be  kept  to  a  practical  minimum 

—  Reduces  cost — thermocouple  grade  wire  should  be  used 
even  on  short  distances 

—  Reduces  resistivity  of  wire  (shorter  lengths  recommended 
by  manufacturer) 

—  Reduces  noise  due  to  electromagnetic  background 

•  Wires  should  be  properly  shielded  and  the  system  should  be 
properly  grounded 
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Temperature 


*  For  full  analysis,  channels  should  be  calibrated  in 
situ  using  a  thermocouple  simulator 

—  Alternately,  each  component  in  the  system  could  be 
calibrated  separately  versus  an  in-situ  system  calibration 

•  Obviously,  this  is  time  consuming 

•  It  assumes  no  feedback  from  any  components  to  other 
components 

—  If  wire  lengths,  junction  box,  etc  are  the  same  then  a  few 
channels  can  be  calibrated  and  the  others  verified  at  one 
or  two  points 

—  While  small,  the  noise  in  the  system  may  not  be  negligible 
so,  at  a  bare  minimum,  a  few  channels  should  be  verified 
at  two  points  to  ensure  uncertainty  is  at  expected  levels 

—  EC-1  calibrated  5  randomly  selected  (in-use)  channels 
yearly  and  verifies  all  channels  prior  to  placing  them  in 
service 
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Temperature 


*  Uncertainty  in  thermocouple  itself  is  currently  the 
largest  contributor  to  combined  uncertainty 

—  EC-1  currently  relies  on  manufacturer’s  provided 
uncertainty  values  and  verifies  a  thermocouple  in  each 
batch  is  within  the  given  uncertainty 

—  Thermocouples  should  be  calibrated;  some  percentage 
are  likely  to  have  lower-than-cited  uncertainty 

—  Selection  of  thermocouple  type  can  be  important 

•  Balance  between  cost  and  accuracy 

•  Match  expected  range  and  thermocouple  within  the  available 
budget 

•  EC-1  uses  Type  E  thermocouples  to  measure  propellant 
temperatures  and  Type  K  to  measure  engine  wall 
temperatures 
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Uncertainty  (°F) 


Temperature 


•  Low  uncertainty  thermocouple  is  1%  of  difference 
between  the  thermocouple  junction  and  reference 

•  High  uncertainty  thermocouple  is  1°C 
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Areas  (Throat) 


*  Throat  diameter  is  measured  using  pin  gauges 

—  Reduces  user-to-user  variation  versus  bore  micrometer 

—  Black  oxide  coating  allows  visual  inspection  instead  of 
frequent  calibration 

—  Smallest  increments  commercially  available  (5x1  O'4  inch) 

—  Not  temperature  controlled  or  monitored  (this  introduces  a 
reasonable  additional  uncertainty) 

—  Main  contributor  to  uncertainty  is  resolution  (increments 
available)  and  user  bias  (strong  function  of  resolution) 

*  Circularity  of  the  nozzle  is  assumed 

—  Not  verified  in  a  rigorous  manner 

—  Requires  careful  nozzle  design 

*  Changes  during  firing  are  also  ignored 

—  Variation  after  run  MUST  be  added  to  uncertainty 

—  Changes  from  run-to-run  indicate  need  to  redesign  nozzle 
cooling  to  lower  uncertainty 
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Areas  (Chamber) 


*  Chamber  is  typically  2  inches  square  with  V4  inch 
radii  at  the  corners 

—  Rounded  corners  should  be  included  in  area  calculation 
(were  neglected  in  early  EC-1  testing) 

*  Dimensions  should  be  measured  before  and  after 
each  series  of  runs 

—  Currently,  EC-1  uses  engineering  tolerances  for  the 
uncertainty  analysis 

—  A  program  to  implement  this  recommendation  and 
augment  uncertainty  accordingly  is  underway 

*  As  with  the  nozzle  area,  changes  during  the 
experiment  are  neglected 

—  No  way  to  measure  these  changes 

—  Heat-sink  hardware  makes  this  unlikely  to  be  true,  so 
appropriate  additions  to  uncertainty  are  being  considered 


Distribution  A:  Approved  for  public  release,  distribution  unlimited 


16 


Uncertainty  (inch2) 


Areas 


Area  Ratio  uncertainty  is  0.55%  of  the  ratio  (area 
ratios  of  9  to  27  investigated) 

The  uncertainty  in  chamber  area  dominates  the  area 
ratio  uncertainty 

—  Currently,  this  uncertainty  is  unrealistically  low  as  cited  on 
previous  slide 
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Throat  Diameter  (inch) 
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Liquid  Mass  Flow  Rate 


*  The  analysis  assumes  a  liquid  fuel  and  gaseous 
oxidizer  as  this  is  typical  in  EC-1 

*  Liquid  flow  is  metered  using  a  cavitating  venturi 

*  Density  and  vapor  pressure  are  important 
components  in  calculating  the  flow  rate 

—  Unless  the  fuel  is  a  single,  well-known  chemical,  they 
should  be  measured  for  each  batch  of  fuel  used 

—  ASTM  standards  (D4052,  D6377)  should  be  used 

—  Unless  provisions  are  taken  to  control  fuel  temperature 
upstream  of  the  venturi,  measurements  should  be  made 
as  a  function  of  temperature 

—  Temperature  enters  the  flow  calculations  through  these 
properties 

*  Pressure  is  the  other  dependent  variable  and  was 
addressed  earlier 
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Liquid  Flow  Rate 


*  Discharge  coefficients  should  be  calibrated  at  least 
yearly  (and  spot-checked  after  periods  of  heavy  use) 

—  Do  not  measure  throat  diameter  and  set  a  discharge 
coefficient  as  this  adds  substantial  uncertainty 

—  Calibrate  using  the  actual  fuels  if  practical;  calibrate  in  the 
test  system  if  practical 

•  Reduces  uncertainty  substantially 

—  Catch-and-weigh  is  generally  the  easiest  calibration  to 
employ  and  can  be  quite  accurate 

•  Use  as  large  a  time  and  weight  as  can  be  practically 
managed 

—  EC-1  calibrated  yearly  in  this  manner 

•  Actual  fuels  are  used  unless  highly  volatile  or  small  quantity 
availability 

•  Water  calibrations  are  also  performed  (used  for  spot-checks 
between  calibrations) 

•  CD  has  been  observed  to  change  on  the  order  of  0.5%  yearly 
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Liquid  Flow  Rates 


*  Sufficient  upstream  and  downstream  distances  must 
be  provided  to  obtain  developed,  not  swirling  flow 

—  EC-1  has  >20D  upstream  and  >13D  downstream 

—  Minimum  industry  recommendations  are  10D  upstream 
and  5D  downstream 

—  Can  be  relaxed  if  calibrating  in  the  test  system;  however, 
discharge  coefficient  will  likely  be  function  of  pressure 

*  Effects  of  dissolved  gases  were  examined  and  found 
to  be  unmeasurable  in  EC-1  system 

—  However,  bubbly  flow  downstream  is  undesirable  for  other 
reasons  so  bladder  tanks  are  used 

—  Fuel  is  not  degassed  prior  to  use 

*  Vapor  pressure  and  discharge  coefficient 
uncertainties  currently  dominate 

—  Prior  to  improvements,  pressure  uncertainty  was  a  large 
contributor 
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Liquid  Flow  Rate 
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Mass  Flow  Rate  (Ib/s) 

The  current  uncertainty  in  liquid  flow  rate  contrasted 
with  the  uncertainty  with  the  current  set-up  except  a 
change  to  the  amplified  transducers 


Distribution  A:  Approved  for  public  release,  distribution  unlimited 


21 


Gas  Flow  Rate 


*  Gas  flow  is  metered  using  a  sonic  nozzle 

*  Dependent  on  pressure  and  temperature 

*  As  with  liquid  flow  rate,  calibrate  to  get  CD 

—  Measuring  area  and  assuming  discharge  coefficient  results 
in  higher  uncertainty 

—  Discharge  coefficients  show  less  change  over  time  than 
the  venturis  (still  has  some  change) 

—  Calibrations  have  been  done  at  CEESI  and  in-line  using  a 
CEESI-calibrated  nozzle  upstream 

•  Previously,  a  catch-and-weigh  setup  has  been  used  but  its 
complexity  and  specialized  equipment  made  it  undesirable 
since  it  did  not  produce  lower  uncertainties 

—  EC-1  recalibrates  in-house  yearly  or  after  heavy  use 
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Gas  Flow  Rate 


*  Use  an  established  procedure  to  calculate  critical 
flow  factor  for  the  measured  temperature  and 
pressure 

—  Several  published  procedures  with  similar,  very  small 
uncertainties  (EC-1  uses  a  procedure  by  Stewart,  Watson 
and  Vaidya) 

—  For  large  throat-to-pipe  ratios  this  should  be  corrected  for 
the  finite  velocity  upstream  of  the  throat 

*  Provide  sufficient  upstream  and  downstream 
distances  to  have  fully  developed,  not  swirling  flow 

—  EC-1  has  >28D  upstream  and  13.5D  downstream 

*  Locate  valves  downstream  to  mitigate  compression 
upstream  of  the  throat 

—  Otherwise  a  nearly  step-wise  change  in  temperature 
overwhelms  the  thermocouple  and  may  prevent  reliable 
temperature  readings 
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Gas  Flow  Rates 


0.08 

0.07 

0.06 

7/7 

1^0.05 

0.04 

t 

<u 

£  0.03 
D 

0.02 

0.01 

8 


□  Current  @40°F 
0  Current  @100°F 
A  Old  Transducers  @40°F 
V  Old  Transducers  @100°F 


6 


0.2  0.3  0.4  0.5  0.6  0.7 


0.3  0.4  0.5  0.6 

Mass  Flow  Rate  (Ib/s) 


0.8 


*  Combined  uncertainty  is  a  larger  percentage  of  flow 
rate  than  that  in  the  liquid  (dominates  uncertainty  in 
mixture  ratio  and  total  mass  flow) 

*  Combined  uncertainty  has  a  strong  dependence  on 
the  uncertainty  in  the  temperature 
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Stoichiometric  Coefficients 


*  CEA  needs  the  formulation  of  the  hydrocarbon  as  an 
input 

*  For  single  component  fuels  this  is  easy,  but  many 
liquid  fuels  are  mixtures  (such  as  RP-1  or  RP-2) 

—  Measurements  should  be  made  for  each  batch  of  fuel 

*  Use  an  ASTM  method  (given  in  standard  D5291)  to 
calculate  the  hydrogen  and  carbon  percentage 

—  While  contaminant  levels  are  low  for  most  fuels,  the 
calculation  should  not  rely  on  hydrogen  percentage  alone 

•  Failure  to  know  the  contaminant  level  introduces  substantial 
additional  uncertainty 

•  Carbon  percentage  is  calculated  by  instruments  at  the  same 
time  a  hydrogen  percentage 

*  Using  a  CXH1  formulation  minimizes  uncertainty  in 
coefficients,  but  does  not  minimize  uncertainty  in 
enthalpy 
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Enthalpy 


*  Again,  for  multicomponent  fuels  the  heat  of 
combustion  should  be  measured  for  every  batch 

—  ASTM  method  D4809  is  recommended 

*  Enthalpy  of  formation  is  calculated  from  this  heat  of 
combustion  along  with  other  well-established  values 

—  These  include  molecular  weights  and  enthalpies  of 
formation  of  water  and  carbon  dioxide 

*  C^Hy  formulation  minimizes  uncertainty  in  enthalpy 

—  This  finding  is  in  direct  opposition  to  that  of  the 
stoichiometric  coefficients 

—  The  sensitivity  coefficients  of  the  CEA  code  are  needed  to 
establish  which  formulation  results  in  a  lower  combined 
uncertainty  in  theoretical  c* 

—  Calculations  at  a  single  operating  condition  indicate  the 
C.,Hy  formulations  is  the  best  choice 
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Theoretical  c* 


*  Analysis  is  based  on  a  single  point 

*  Rough  sensitivity  coefficient  determination  around 
that  single  point 

*  Mixture  ratio  dominates  the  combined  uncertainty 

—  This  finding  is  due  in  large  part  to  its  sensitivity 

—  Mixture  ratio  uncertainty  is  dominated  by  uncertainty  in 
gas  flow  rate  which  is,  in  turn,  dominated  by  uncertainty  in 
temperature 

•  From  this  EC-1  has  found  it  imperative  to  calibrate 
thermocouples  prior  to  use 

*  At  single  condition  considered  (MR=2.4,  AR=21.4, 
Pc=700  psi,  T=77°F)  the  uncertainty  of  0.14%  using 
the  lowest  thermocouple  uncertainty  (0.30%  for 
other  thermocouple  manufacturer) 
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Measured  c* 


*  Analysis  based  on  a  range  of  points  but  a  single 
venturi  and  sonic  nozzle 

—  Chamber  pressures  300-1000  psi,  Throat  Diameters  0.45 
to  0.65  inch,  total  mass  flow  rates  0.25-1.15  Ib/s 

*  Mass  flow  rate  dominates  the  combined  uncertainty 

—  Main  component  of  the  total  mass  flow  rate  uncertainty  is 
the  gas  flow  rate 

—  Again,  temperature  is  the  major  contributor 

*  Dependent  on  temperatures,  using  the  low 
uncertainty  thermocouples,  the  combined 
uncertainty  in  c*  could  be  as  high  as  7.6%  or  as  low 
at  0.5% 

—  For  a  moderate  (and  typical)  temperature  of  70°F  the 
uncertainty  is  <2% 
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c*-Efficiency 


Only  a  single  condition  has  been  considered  to  date 
(mlia=0.208  Ib/s,  m=0.500  Ib/s,  AR=21.4,  Pc=700  psi, 
T=77°F) 

Combined  uncertainty  at  this  condition  is  1.22% 

—  Strongly  driven  by  temperature  uncertainty 

—  Due  to  strong  gas  flow  rate  dependence,  combined 
uncertainty  is  close  to  specific  uncertainty  in  gas  flow  rate 

•  This  suggests  that  the  range  for  40-1 00°F  may  run  from 
nearly  8%  to  under  1% 

Updates  to  system  have  decreased  uncertainty 
significantly 

—  At  specific  condition  cited  just  changing  transducers 
HALVED  the  uncertainty 

—  Prior  uncertainty  was  just  under  3% 

Does  not  include  heat-loss  corrections  or 
uncertainties 

Distribution  A:  Approved  for  public  release,  distribution  unlimited 


29 


Future  Work 


*  Several  items  remain  to  further  clarify  uncertainty  or 
to  improve  the  combined  uncertainty 

*  First  and  foremost,  thermocouples  will  be  calibrated! 

*  Heat  transfer  losses  to  the  heat-sink  hardware 
remain  to  be  evaluated 

*  A  more  robust  and  throrough  evaluation  of  the  CEA 
code  at  the  range  of  typical  conditions 

*  On-going  monitoring  and  assessment  of  engine 
cross  sectional  area  is  being  implemented 

*  Determine  temperature  dependence  of  the  pressure 
transducers  through  calibration  in  an  oven 

*  Confidence  bounds  have  not  been  established 

—  Unlikely  that  a  95%  confidence  bound  will  be  a  simple 
multiple  of  2  (too  few  degrees  of  freedom) 
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